Testing Distribution Identity Efficiently

We consider the problem of testing distribution identity. Given a sequence of independent samples from an unknown distribution on a domain of size n, the goal is to check if the unknown distribution approximately equals a known distribution on the same domain. While Batu, Fortnow, Fischer, Kumar, Rubinfeld, and White (FOCS 2001) proved that the sample complexity of the problem is Õ( √ n ·poly(1/ε)), the running time of their tester is much higher: O(n)+ Õ( √ n · poly(1/ε)). We modify their tester to achieve a running time of Õ( √ n ·poly(1/ε)). Let p and q be two probability distributions on [n]1, and let ‖p− q‖1 denote the l1-distance between p and q. In this paper, algorithms have access to two distributions q and p. • The distribution p is known: for each i ∈ [n], the algorithm can query the probability pi of i in constant time. • The distribution q is unknown: the algorithm can only obtain an independent sample from q in constant time. An identity tester is an algorithm such that: • if p = q, then it accepts with probability 2/3, • if ‖p−q‖1 ≥ ε, then it rejects with probability 2/3. Batu, Fortnow, Fischer, Kumar, Rubinfeld, and White [BFF+01] proved that there is an identity tester that uses only Õ( √ n · poly(1/ε)) samples from q. A shortcoming of their algorithm is a running time of O(n)+ Õ( √ n ·poly(1/ε)). In this note, we show that their tester can be modified to achieve a running time of Õ( √ n ·poly(1/ε)). It is also well known that Ω(√n) samples are required to tell the uniform distribution on [n] from a distribution that is uniform on a random subset of [n] of size n/2. 1 The Original Tester We now describe the tester of Batu et al. [BFF+01], which is outlined as Algorithm 1. Let ε′ = ε/C, where C is a sufficiently large positive constant. The tester starts by partitioning the set [n] into k + 1 =